Decoding device, reception device, transmission device, transmission/reception system, decoding method, and storage medium having decoding program stored therein

ABSTRACT

A method for receiving data, the method includes receiving a plurality of assets, each asset including a Media Processing Unit (MPU); receiving a packet identification for each asset among the plurality of assets via respective Moving Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packets; receiving a time stamp information referencing UTC (Coordinated Universal Time) in the MPU, the time stamp information representing a presentation time of a first access unit in presentation order in the MPU; and receiving a leap second insertion flag indicating when a leap second will be inserted, and receiving a leap second deletion flag indicating when a leap second will be deleted, the leap second insertion flag and the leap second deletion flag indicating a presence of a leap second event, wherein the first access unit in the MPU is presented at the time signaled in reference to the time stamp information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/031,736, filed Jul. 10, 2018, which is a continuation of U.S.application Ser. No. 15/550,043, filed on Aug. 10, 2017, now U.S. Pat.No. 10,194,196, issued Jan. 29, 2019, which is a National Stage ofInternational Application No. PCT/JP2016/000874, filed on Feb. 18, 2016,which claims priority from Japanese Patent Application No. 2015-040456,filed on Mar. 2, 2015, the contents of all of which are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present invention relates to a decoding apparatus, a receiver, atransmitter, a transmission/reception system, a decoding method, and astorage medium storing a decoding program for transmission of encodedpackets.

BACKGROUND ART

Digitization that changes signals transmitted/received in televisionbroadcasting from analog signals to digital signals has proceeded aroundthe world. In Japan, terrestrial digital broadcasting based on theISDB-T (Integrated Services Digital Broadcasting-Terrestrial) scheme isperformed.

In the terrestrial digital broadcasting based on the ISDB-T scheme,television program information including video information and audioinformation and data broadcast information (hereinafter, referred to astelevision program information and the like) are transmitted by radiowaves to a large number of receivers. Further, also in satellitebroadcasting and cable television broadcasting, similar information istransmitted. The receivers reproduce video and audio on the basis of thetransmitted television program information and the like.

Further, for reasons of enhancing the quality of video and audio to beoutput to receivers, increasing the number of types, and the like, ithas been investigated that a larger number of pieces of information aretransmitted to receivers, and standardization of a new media transportscheme referred to as MMT (MPEG (Moving Picture Experts Group) MediaTransport) has proceeded, as described in, for example, NPL 1.

In PTL 1, a method for appropriately decoding coded data in MMT isdescribed.

CITATION LIST Patent Literature

[PTL 1] International Publication WO2014/196189

Non Patent Literature

[NPL 1] “MMT-Based Media Transport Scheme in Digital BroadcastingSystems, Version 1.1”, [online], December 2014, Association of RadioIndustries and Businesses, [retrieved on Feb. 26, 2015], the Internet<URL:http://arib.or.jp/english/html/overview/doc/2-STD-B60v_1_1.pdf>

SUMMARY OF INVENTION Technical Problem

However, even when television program information and the like aretransmitted by radio waves, it is difficult for a receiver toappropriately receive the television program information and the like insome cases due to changes in weather conditions or the like.

Therefore, in MMT, via a plurality of transmission paths referred to asa communication line and radio waves, television program information andthe like are transmitted/received, and when it has been difficult toappropriately receive television program information and the liketransmitted by radio waves, video and audio are reproduced on the basisof television program information and the like transmitted via acommunication line. To perform reproduction in such a manner, it isnecessary for the receiver to decode and reproduce the televisionprogram information and the like transmitted via the communication lineand the television program information and the like transmitted by radiowaves at appropriate timings.

Here, a broadcaster generates time information so that a receiver canperform reproduction at appropriate timings on the basis of a time stampof NTP (Network Time Protocol) and transmits the generated timeinformation by being associated with television program information andthe like. Now, the time stamp of NTP refers to information indicating atime based on NTP. The receiver reproduces video and audio based on thetelevision program information and the like at appropriate timings onthe basis of a separately acquired time stamp of NTP and the timeinformation associated with the television program information and thelike.

The time stamp of NTP is based on Coordinated Universal Time.Coordinated Universal Time is adjusted to be made close to UniversalTime once in several years. The adjustment is performed by inserting aleap second or eliminating a leap second. Specifically, when a leapsecond is inserted, a time stamp indicating, for example, eightfifty-nine and fifty nine seconds (hereinafter, referred to as “8:59:59”or the like) is generated twice continuously. Further, when a leapsecond is eliminated, a time stamp indicating the next second of 8:59:58is generated so as to indicate 9:00:00.

Then, in the method described in Patent Literature 1, when theadjustment is performed, it is difficult for a receiver to appropriatelydetermine timings of reproducing video and audio based on transmittedtelevision program information and the like. In Non Patent Literature 1,any response to the case in which the adjustment is performed is notdisclosed.

Further, even when the adjustment is not performed, it is difficult thata receiver having received television program information and the liketransmitted via a communication line and television program informationand the like transmitted by radio waves reproduces video and audio bydecoding these pieces of information at appropriate timings.

Furthermore, even in the case of not performing the adjustment, when areceiver receives television program information and the like in anorder different from a transmission order upon transmitting thetelevision program information and the like via a communication line, itis difficult that the receiver decodes and reproduces these pieces ofinformation at appropriate timings.

Therefore, the present invention is intended to provide a decodingapparatus, a receiver, a transmitter, a transmission/reception system, adecoding method, and a storage medium storing a decoding program fordecoding pieces of information sequentially transmitted at appropriatetimings.

Solution to Problem

A decoding apparatus according to an exemplary aspect of the presentinvention includes: a decoding unit for decoding packets transmittedfrom a transmitter; and a decode time determination unit for determiningtimings of decoding the packets by the decoding unit on the basis ofadjustment information in accordance with adjustment of times forcausing Coordinated Universal Time to be close to Universal Time, thetimes being included in the packets, wherein the packets includeprocessing interval information that indicates a processing interval ofaccess units that are sets including own packets, and the decode timedetermination unit determines to sequentially decode the packetsconfiguring the respective access units at intervals in accordance withthe interval indicated by the processing interval information when theadjustment information indicates that the times are adjusted.

A receiver according to an exemplary aspect of the present inventionincludes a decoding apparatus according to any one of aspects and areception apparatus that receives packets.

A transmitter according to an exemplary aspect of the present inventionincludes, on a reception side, a transmission unit for transmittingpackets including adjustment information in accordance with adjustmentof times for causing Coordinated Universal Time to be close to UniversalTime in order to decode packets in accordance with contents atpredetermined timings and processing interval information that indicatesa processing interval of access units that are sets including ownpackets.

A transmission/reception system according to an exemplary aspect of thepresent invention includes a receiver and a transmitter according to anyone of aspects.

A decoding method according to an exemplary aspect of the presentinvention includes: a decoding step of decoding packets transmitted froma transmitter; and a decode time determination step of determiningtimings of decoding the packets in the decoding step on the basis ofadjustment information in accordance with adjustment of times forcausing Coordinated Universal Time to be close to Universal Time, thetimes being included in the packets, wherein the packets includeprocessing interval information that indicates a processing interval ofaccess units that are sets including own packets, and wherein the decodetime determination step determining to sequentially decode the packetsconfiguring the respective access units at intervals in accordance withthe interval indicated by the processing interval information when theadjustment information indicates that the times are adjusted.

A storage medium according to an exemplary aspect of the presentinvention stores a decoding program that causes a computer to execute:decoding processing for decoding packets transmitted from a transmitter;and decode time determination processing for determining timings ofdecoding the packets in the decoding processing on the basis ofadjustment information in accordance with adjustment of times forcausing Coordinated Universal Time to be close to Universal Time, thetimes being included in the packets, wherein the packets includeprocessing interval information that indicates a processing interval ofaccess units that are sets including own packets, and wherein the decodetime determination processing determines to sequentially decode thepackets configuring the respective access units at intervals inaccordance with the interval indicated by the processing intervalinformation when the adjustment information indicates that the times areadjusted.

Advantageous Effects of Invention

According to the present invention, pieces of information sequentiallytransmitted can be decoded at appropriate timings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of atransmission/reception system of a first example embodiment of thepresent invention.

FIG. 2 is a protocol stack diagram illustrating a protocol configurationof multimedia information transmitted/received by radio waves.

FIG. 3 is a protocol stack diagram illustrating a protocol configurationof multimedia information transmitted/received via a communication line.

FIG. 4 is a block diagram illustrating a configuration example of areception buffer in a receiver.

FIG. 5 is an illustrative diagram illustrating an outline of accessunits in an MPU, and presentation times and decoding times thereof.

FIG. 6 is an illustrative diagram illustrating an example of MPUstransmitted/received when a leap second is inserted.

FIG. 7 is an illustrative diagram illustrating an example in which areceiver processes MPUs transmitted/received when a leap second isinserted.

FIG. 8 is a flowchart illustrating an example in which a receiverprocesses MPUs transmitted/received when a leap second is inserted.

FIG. 9 is an illustrative diagram illustrating an example of MPUstransmitted/received when a leap second is eliminated.

FIG. 10 is an illustrative diagram illustrating an example in which areceiver processes MPUs transmitted/received when a leap second iseliminated.

FIG. 11 is a flowchart illustrating an example in which a receiverprocesses MPUs transmitted/received when a leap second is eliminated.

FIG. 12 is a block diagram illustrating a configuration example of adecoding apparatus of a second example embodiment of the presentinvention.

DESCRIPTION OF EMBODIMENTS A First Example Embodiment

A transmission/reception system of a first example embodiment of thepresent invention will be described with reference to the accompanyingdrawings. FIG. 1 is a block diagram illustrating a configuration exampleof the transmission/reception system of the first example embodiment ofthe present invention. As illustrated in FIG. 1, thetransmission/reception system of the first example embodiment of thepresent invention includes: a transmitter 100 that transmitsinformation; and a receiver 200 that receives information. Hereinafter,the transmitter 100 and the receiver 200 may be realized by, forexample, a computer such as a CPU (Central Processing Unit) or the likemounted with a single circuit or a plurality of circuits, the computerexecuting processing in accordance with program control. Specifically,the transmitter 100 and the receiver 200 are mounted with, for example,software for realizing respective operations described below. Further,the transmitter 100 and the receiver 200 may be configured to realizethe respective operations described below by executing processing inaccordance with program control of the software.

Hereinafter, description will be made, assuming that a leap second thatis a time to be inserted or eliminated is one second, but the time maybe shorter or longer than one second.

The transmitter 100 is installed by, for example, a broadcaster andtransmits television program information and the like. The receiver 200is installed by, for example, a viewer who views video and audio basedon television program information and the like and receives thetelevision program information and the like transmitted by thetransmitter 100. Then, the receiver 200 presents (referred to also asreproduces or outputs) video and audio based on the received televisionprogram information and the like. Here, the video and audio presented inthe receiver 200 include information of application software including aso-called app and various pieces of information such as captions, data,and the like. Now, an apparatus that receives information transmitted bythe transmitter 100 and an apparatus that decodes the informationreceived by the apparatus may be provided separately. Further,separately from the apparatus that decodes the information, an apparatusthat presents video, audio, and the like based on the decodedinformation may be provided. In other words, means for realizing therespective operations described below may be apparatuses separate fromeach other.

As illustrated in FIG. 1, the transmitter 100 transmits multimediainformation 600 a by radio waves via an antenna 300. Further, thetransmitter 100 transmits multimedia information 600 b via acommunication line (which may be a communication network such as theInternet or the like) 400. Here, the receiver 200 receives themultimedia information 600 a via an antenna 500. Further, the receiver200 receives the multimedia information 600 b via the communication line400. The pieces of multimedia information 600 a and 600 b each areinformation including television program information and the like, anddetails thereof will be described later.

Now, the receiver 200 includes, for example, a reception unit 210 thatreceives pieces of multimedia information 600 a and 600 b and a decodingunit 220 that executes decoding processing to be described later.

Pieces of multimedia information 600 a and 600 b transmitted/receivedbetween the transmitter 100 and the receiver 200 will be describedbelow.

FIG. 2 is a protocol stack diagram illustrating a protocol configurationof multimedia information 600 a transmitted/received by radio waves. Asillustrated in FIG. 2, the multimedia information 600 a includes an IP(Internet Protocol) packet of the TLV (Type Length Value) packet formatand TMCC (Transmission and Multiplexing Configuration and Control) thatis a control signal.

Further, the IP packet of the TLV format (hereinafter, referred to alsoas the TLV packet) includes time information based on NTP and televisionprogram information and the like based on MMT.

FIG. 3 is a protocol stack diagram illustrating a protocol configurationof multimedia information 600 b transmitted/received via thecommunication line 400. As illustrated in FIG. 3, the multimediainformation 600 b includes an IP packet in accordance with televisionprogram information and the like and time information based on NTP.Further, the IP packet includes television program information and thelike based on MMT.

Video information and audio information of the television programinformation and the like included in the pieces of multimediainformation 600 a and 600 b are described below. Here, the videoinformation refers to information based on video signals. Further, theaudio information refers to information based on audio signals.Furthermore, the transmitter 100 encodes video signals and audio signalsand stores the coded signals in an MFU (Media Fragment Unit). Then, datafor, for example, 1 GOP (Group Of Picture) based on a plurality of MFUsconfigures an MPU (Media Processing Unit), and the transmitter 100stores MFUs in an MMTP (MMT Protocol) payload and transmits the MFUs asan IP packet by an MMTP packet. The video signals and the audio signalsare converted into various types of formats by coding or packetization,but in the present example, information based on these video signals andinformation based on these audio signals to be changed to various typesof formats will be collectively referred to as video information andaudio information. Further, information including video information andaudio information to be presented to a viewer will be referred to alsoas content information.

The MMTP packet includes a header portion referred to as an MMTP headerand a payload portion referred to as an MMTP payload. The contentinformation is included in the MMTP payload. The transmitter 100generates an MMTP header including time stamp information that isinformation indicating a time based on an NTP counter.

Further, the MMTP payload stores MPU_presentation_time information thatindicates a time (hereinafter, referred to as a presentation time) ofcausing the receiver 200 to output video and audio based on contentinformation included in an MMTP packet including own information. Thereceiver 200 executes processing so as to output the video and audiobased on the content information at the presentation time indicated bythe MPU_presentation_time information. Here, the presentation time isexpressed in an NTP time stamp format in this example, but thepresentation time may be expressed in other format which can express thetime.

The MMTP payload stores mpu_sequence_number information. Thempu_sequence_number information refers to information that indicates anmpu_sequence_number that is a sequence number assigned to identify eachMPU from other MPUs for each same asset.

Further, the MMTP payload stores packet ID information. The packet IDinformation refers to information that indicates a packet ID foridentifying, for example, an MMTP packet configuring one asset from anMMTP packet configuring another asset.

Here, for the time information, a flag in accordance with adjustment forcausing Coordinated Universal Time to be close to Universal Time isprepared. Specifically, when, for example, a leap second is inserted,for time information transmitted during a predetermined period of timeprior to the timing of timing of inserting the leap second, a leapsecond insertion flag in accordance with insertion of a leap second isset.

Further, when, for example, a leap second is eliminated, for timeinformation transmitted during a predetermined period of time prior tothe timing of eliminating the leap second, a leap second eliminationflag in accordance with elimination of a leap second is set. Here, thepredetermined period of time is, for example, one month. Further, theleap second insertion flag and the leap second elimination flag areequivalent to adjustment information. Furthermore, a timing of setting aflag, a reset timing, and a type of a flag to be set are not limitedthereto, and a flag in accordance with another timing or another typemay be used.

Next, a configuration of the receiver 200 will be described. FIG. 4 is ablock diagram illustrating a configuration example of a reception bufferin the receiver 200.

In the example illustrated in FIG. 4, the reception buffer of thereceiver 200 includes a TLV packet buffer 201, an IP packet buffer 202,an MMTP transport buffer 203, an MMTP buffer 204, a predecoding buffer205, a decoder 206, and a decoded picture buffer 207.

A TLV packet extracted from multimedia information 600 a transmitted byradio waves is input to the TLV packet buffer 201 and temporality storedtherein.

An IP packet based on the TLV packet temporarily stored in the TLVpacket buffer 201 is input to the IP packet buffer 202 and temporarilystored therein. Further, an IP packet of multimedia information 600 btransmitted from the transmitter 100 is input to the IP packet buffer202 via the communication line 400 and temporarily stored therein.

In the MMTP transport buffer 203, the IP packet, being changed to MMTP,temporarily stored in the IP packet buffer 202 is classified withrespect to each asset such as video, audio, and the like on the basis ofpacket ID information included in an MMTP header. Then, the classifiedIP packets are input to the MMTP buffer 204 in accordance with theclassification result.

In the example illustrated in FIG. 4, the MMTP buffer 204 includes MMTPB1, MMTP Bn, and MMTP Bs. Further, an IP packet of an asset for video isinput to MMTP B1. Furthermore, an IP packet of an asset for audio isinput to MMTP Bn. Moreover, an IP packet of an asset for a controlmessage is input to MMTP Bs and temporarily stored therein.

The IP packet of an asset for video input to MMTP B1 is restored to dataof an NAL (Network Abstraction Layer) unit and temporarily stored inMMTP B1.

The IP packet of an asset for audio input to MMTP Bn is restored to dataof the LATM (Low-overhead MPEG-4 Audio Transport Multiplex)/LOAS (LowOverhead Audio Stream) format and temporarily stored in MMTP Bn. Herein,description is made, assuming that the IP packet of an asset for audiois restored to data of the LATM/LOAS format as one example, but aconfiguration may be made so that restoration to data of another formatis performed.

Then, each piece of data and the control message temporarily stored inthe MMTP buffer 204 are input to the predecoding buffer 205.

As illustrated in FIG. 4, the predecoding buffer 205 includes anelementary stream buffer EB1 for an elementary stream 1, a main bufferBn for an elementary stream n, and a main buffer Bs for systeminformation.

The data of the NAL unit temporarily stored in MMTP B1 is input to theelementary stream buffer EB1. Here, the elementary stream buffer EB1 is,for example, a CPB (Coded Picture Buffer).

The data of the LATM/LOAS format temporarily stored in MMTP Bn is inputto the main buffer Bn and temporarily stored therein. Here, data of theRaw Data Stream format may be input to the main buffer Bn andtemporarily stored therein.

The control message temporarily stored in MMTP Bs is input to the mainbuffer Bs and temporarily stored therein. Herein, description is made,assuming that data of the LATM/LOAS format as one example is restored todata of the Raw Data Stream format as one example, but a configurationmay be made so that data of another format is restored to data offurther another format.

Each piece of data and the control message input to the predecodingbuffer 205 are output to the decoder 206 and decoded at a timing inaccordance with a DTS (Decode Time Stamp), and temporarily stored. TheDTS will be described later.

As illustrated in FIG. 4, the decoder 206 includes a video decoder D1,an audio decoder Dn, and a system decoder Ds.

The data output from the elementary stream buffer EB1 to the decoder 206is decoded to video data by the video decoder D1. Then, the decodedvideo data is input, as needed, to the decoded picture buffer 207 andtemporarily stored therein.

Further, the data output from the main buffer Bn to the decoder 206 isdecoded to audio data by the audio decoder Dn.

Further, the control message output from the main buffer Bs to thedecoder 206 is decoded to control information by the system decoder Ds.Furthermore, the control information is used to control each unit in thereceiver 200.

The DTS (referred to also as the decode time) is described below. Apresentation time and a decode time of video signals or audio signalsare provided using an MPU time stamp descriptor and an MPU extended timestamp descriptor.

FIG. 5 is an illustrative diagram illustrating an outline of accessunits in an MPU, and presentation times and decoding times thereof. FIG.5 schematically illustrates bit streams configuring the MPU forrespective access units that are units of the same presentation time. Inthe example illustrated in FIG. 5, a bit stream configuring each accessunit configures a B frame and an I frame or a B frame (hereinafter,described as an I/B frame). Here, In FIG. 5, IRAP (Intra Random AccessPoint) refers to a bit stream at a timing when coding is started.Further, in the example illustrated in FIG. 5, an I/B frame is disposedat intervals of seven frames, and frames therebetween are B frames.

In the example illustrated in FIG. 5, each access unit is previouslyassigned with a number in a presentation order. In FIG. 5, using numbersassigned to “B,” “I/B,” and “IRAP,” a presentation order of therespective access units is illustrated. Further, the example illustratedin the upper side of FIG. 5 illustrates that the respective access unitshave been decoded in an order different from the presentation order.

As described above, the MMTP payload stores MPU_presentation_timeinformation. Using the MPU_presentation_time information, a presentationtime of an access unit to be presented first in the MPU is indicated inthe NTP time stamp format. Therefore, in FIG. 5, video based on a Bframe of a 25th access unit is presented at a time indicated by theMPU_presentation_time information. Here, the MPU_presentation_timeinformation is equivalent to presentation time information.

Further, a DTS of an nth access unit in an MPU is calculated by thefollowing equation.

DTS(n)=MPU_Presentation_time−MPU_decoding_time_offset/timescale+Σpts_offset(k)/timescale  (1)

wherein MPU_Presentation_time represents a time indicated in the NTPtime stamp format by MPU_presentation_time information,MPU_decoding_time offset represents a difference value between a decodetime of an access unit decoded first and a presentation time of anaccess unit presented first, pts_offset represents a difference valuebetween a presentation time of an immediately prior access unit and apresentation time of the current access unit in a presentation order inthe same MPU, timescale represents a value indicating a unit of thesetimes, and k represents a number of access units in the MPU. Thepts_offset is equivalent to processing interval information.

Here, a PTS (Presentation Time Stamp) that is a presentation time of annth access unit in an MPU is calculated by the following equation.

PTS(n)=DTS(n)+dts_pts_offset(n)/timescale   (2)

wherein dts_pts_offset(n) represents a difference value between a decodetime and a presentation time of an nth access unit in an MPU.

The transmitter 100 transmits each piece of information so as to bereceived by the receiver 200 before a start of decoding of an MPU in thereceiver 200 so that the receiver 200 can calculate a DTS and a PTSusing equation (1) and equation (2).

Next, operations of the transmitter 100 and the receiver 200 in theexample embodiment of the present invention will be described.Initially, operations of the transmitter 100 and the receiver 200 uponinserting a leap second in order to cause Coordinated Universal Time tobe close to Universal Time are described below.

FIG. 6 is an illustrative diagram illustrating an example of MPUstransmitted/received when a leap second is inserted.

In the present example, it is assumed that the respective MPUs aresequentially transmitted at intervals of 0.5 seconds. Then, in theexample illustrated in FIG. 6, at 8:59:57. 0, respective MMTP packets ofan MPU in which a value indicated by mpu_sequence_number information(indicated as MPU sequence_number in FIG. 6) is “1” are transmitted.Further, at 8:59:57. 5, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “2” aretransmitted. At 8:59:58. 0, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “3” aretransmitted. Further, at 8:59:58. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “4” aretransmitted. At 8:59:59. 0 of a first time, respective MMTP packets ofan MPU in which a value indicated by mpu_sequence_number information is“5” are transmitted. Further, at 8:59:59. 5 of a first time, respectiveMMTP packets of an MPU in which a value indicated by mpu_sequence_numberinformation is “6” are transmitted. At 8:59:59. 0 of a second time,respective MMTP packets of an MPU in which a value indicated bympu_sequence_number information is “7” are transmitted. Further, at8:59:59. 5 of a second time, respective MMTP packets of an MPU in whicha value indicated by mpu_sequence_number information is “8” aretransmitted. At 9:00:00. 0, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “9” aretransmitted. Further, at 9:00:00. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “10”are transmitted.

Here, it is assumed that a value in accordance with a presentation timeindicated by MPU_presentation_time information indicates a value inaccordance with a time after two seconds from a transmission time.

Then, in the example illustrated in FIG. 6, a presentation time inaccordance with a value indicated by MPU_presentation_time informationof MMTP packets of an MPU in which a value indicated bympu_sequence_number information is “1” is 8:59:59. 0. Further, apresentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “2” is 8:59:59. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “3” is 9:00:00. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “4” is 9:00:00. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “5” is 9:00:01. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “6” is 9:00:01. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “7” is 9:00:01. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “8” is 9:00:01. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “9” is 9:00:02. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “10” is 9:00:02.5.

Therefore, with respect to the MMTP packets of the MPU in which a valueindicated by mpu_sequence_number information is “5” and the MMTP packetsof the MPU in which a value indicated by mpu_sequence_number informationis “7,” each of the presentation times in accordance with a valueindicated by MPU_presentation_time information is 9:00:01. 0. Therefore,there are two MPUs in which a presentation time in accordance with avalue indicated by MPU_presentation_time information is 9:00:01. 0.

Further, with respect to the MMTP packets of the MPU in which a valueindicated by mpu_sequence_number information is “6” and the MMTP packetsof the MPU in which a value indicated by mpu_sequence_number informationis “8,” each of the presentation times in accordance with a valueindicated by MPU_presentation_time information is 9:00:01. 5. Therefore,there are two MPUs in which a presentation time in accordance with avalue indicated by MPU_presentation_time information is 9:00:01. 5.

In contrast, due to insertion of 8:59:59 as a leap second, whileprocessing from 8:59:59. 0 to 8:59:59. 999 has been repeated twice,there are only two MPUs in which a presentation time in accordance witha value indicated by MPU_presentation_time information is a time between8:59:59. 0 and 8:59:59. 999. Then, there is no MPU used for presentationbetween 8:59:59. 0 and 8:59:59. 999 of a second time.

In other words, when processing timings of MPUs used for presentationare determined on the basis of set MPU_presentation_time information, aproblem may occur upon inserting a leap second, and it is not possiblefor the invention described in Patent Literature 1 to solve such aproblem.

FIG. 7 is an illustrative diagram illustrating an example in which thereceiver 200 processes MPUs transmitted/received when a leap second isinserted. FIG. 8 is a flowchart illustrating an example in which thereceiver 200 processes MPUs transmitted/received when a leap second isinserted.

In the example illustrated in FIG. 7, MMTP packets of MPUs in whichvalues indicated by mpu_sequence_number information are 1 to 10 aretransmitted. When a leap second insertion flag is set for timeinformation included in the MMTP packets (Y of step S101) and a timingof inserting a leap second falls within a predetermined time (Y of stepS102), the receiver 200 moves to processing of step S103, and when theabove case is not realized (N of step S101 or N of step S102), thereceiver 200 moves to processing of step S108. In the processing of stepS108, the receiver 200 executes normal processing that processes MMTPpackets used for presentation at timings based on MPU_presentation_timeinformation (step S108).

Specifically, in the processing of step S108, the receiver 200 decodesand presents the respective MMTP packets at timings based on calculationresults using equation (1) and equation (2) described above on the basisof the MPU_presentation_time information.

Further, the receiver 200 determines to execute processing such asdecoding and the like on the basis of timings determined in aself-running mode in processing of step S103 (step S103).

Here, the leap second insertion flag is indicated as NTP leap indicatorin FIGS. 6 and 7 and it is reset at 9:00:00. 0. Specifically, forexample, a leap second insertion flag is set for time informationincluded in MMTP packets transmitted before 9:00:00. 0, but no leapsecond insertion flag is set for time information included in MMTPpackets transmitted after 9:00:00. 0.

Hereinafter, the self-running mode is described. The self-running moderefers to a mode in which the receiver 200 determines timings ofdecoding MMTP packets of each MPU without referring toMPU_presentation_time information.

The receiver 200 determines timings of decoding the MMTP packets by thefollowing processing in the self-running mode (step S104). Specifically,the receiver 200 sequentially determines, at intervals in accordancewith a value of pts_offset used in the calculation of equation (1) inthe processing of step S108 of an immediately prior stage, timings ofdecoding subsequent access units. More specifically, timings of decodingrespective MMTP packets configuring the access units are determined soas to decode MMPT packets configuring the access units at time intervalsof a value of pts_offset, for example.

Then, the receiver 200 decodes the respective MMTP packets configuringthe access units on the basis of the timings determined in theprocessing of step S104 (step S105). The unit of decoding may be anaccess unit or another unit.

The receiver 200 presents video and audio based on the MMTP packetsdecoded in the processing of step S105 (step S106). Specifically, atintervals in accordance with the value of pts_offset used in thecalculation of equation (1) in the processing of step S108 of animmediately prior stage, timings of presenting video and audio inaccordance with subsequent access units are sequentially determined.More specifically, timings of presenting video and audio in accordancewith respective MMTP packets configuring the access units are determinedso as to present video and audio in accordance with MMTP packetsconfiguring the access units at time intervals of a value of pts_offset,for example. Then, the receiver 200 presents the video and audio at thedetermined timings.

When a predetermined time has not passed from insertion of a leap second(N of step S107), the receiver 200 moves to the processing of step S104.

Then, by repeating the processing of steps S104 to S106, the MMTPpackets configuring the respective access units are decoded at timingsin accordance with the value of pts_offset in an order of sequencenumbers indicated by mpu_sequence_number information, as illustrated inthe lower side of FIG. 7. Then, video and the like based on therespective MMTP packets are sequentially presented.

It should be noted that in general, in a series of MPUs, a value ofpts_offset is not changed. Therefore, when on the basis of a calculationresult in processing of step S108 of the last time and a value ofpts_offset used in the calculation, MMTP packets of access units fromthe next time are used to determine decode timings, it is possible thatthe respective MMTP packets are decoded at appropriate timings and videoand the like based on the MMTP packets are sequentially presented.

When the predetermined time has passed from insertion of the leap second(Y of step S 107), the receiver 200 moves to normal processing fordetermining timings of MMTP packets used for presentation on the basisof MPU_presentation_time information (step S108).

The determination in the processing of step S107 may be performed on thebasis of whether a predetermined time has passed from a time of settingno leap second insertion flag.

Here, “the predetermined time” in the processing of steps S102 and S107is, for example, three seconds. Therefore, the processing of steps S104to S106 is repeated over three seconds each before and after a timing ofinserting a leap second. Specifically, the processing of steps S104 toS106 is repeated, for example, from 8:59:57 that is three seconds before9:00:00 when a leap second is inserted to 9:00:03 that is three secondsafter 9:00:00 when the leap second has been inserted.

Therefore, in the present example, the receiver 200 executes processingbased on the self-running mode over three seconds before and after atiming of inserting a leap second and executes normal processing of stepS108 for a period other than the above. It should be noted that the timeand the number of seconds used for the description in the presentexample are illustrative, and another time and another number of secondsmay be used.

Next, operations of the transmitter 100 and the receiver 200 uponeliminating a leap second to cause Coordinated Universal Time to beclose to Universal Time will be described. FIG. 9 is an illustrativediagram illustrating an example of MPUs transmitted/received when a leapsecond is eliminated.

In the example illustrated in FIG. 9, it is assumed that respective MPUsare sequentially transmitted at intervals of 0.5 seconds. Then, in theexample illustrated in FIG. 9, at 8:59:56. 0, respective MMTP packets ofan MPU in which a value indicated by mpu_sequence_number information is“1” are transmitted. Further, at 8:59:56. 5, respective MMTP packets ofan MPU in which a value indicated by mpu_sequence_number information is“2” are transmitted. At 8:59:57. 0, respective MMTP packets of an MPU inwhich a value indicated by mpu_sequence_number information is “3” aretransmitted. Further, at 8:59:57. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “4” aretransmitted. At 8:59:58. 0, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “5” aretransmitted. Further, at 8:59:58. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “6” aretransmitted. At 9:00:00. 0, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “7” aretransmitted. Further, at 9:00:00. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “8” aretransmitted. At 9:00:01. 0, respective MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “9” aretransmitted. Further, at 9:00:01. 5, respective MMTP packets of an MPUin which a value indicated by mpu_sequence_number information is “10”are transmitted.

It is assumed that a value in accordance with a presentation timeindicated by MPU_presentation_time information indicates a value inaccordance with a time after two seconds from a transmission time.

Then, in the example illustrated in FIG. 9, a presentation time inaccordance with a value indicated by MPU_presentation_time informationof MMTP packets of an MPU in which a value indicated bympu_sequence_number information is “1” is 8:59:58. 0. Further, apresentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “2” is 8:59:58. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “3” is 8:59:59. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “4” is 8:59:59. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “5” is 9:00:00. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “6” is 9:00:00. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “7” is 9:00:02. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “8” is 9:00:02. 5.A presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “9” is 9:00:03. 0.Further, a presentation time in accordance with a value indicated byMPU_presentation_time information of MMTP packets of an MPU in which avalue indicated by mpu_sequence_number information is “10” is 9:00:03.5.

Therefore, there is not either 8:59:59. 0 that is a presentation time inaccordance with the value indicated by the MPU_presentation_timeinformation of the MMTP packets of the MPU in which a value indicated bympu_sequence_number information is “3” or 8:59:59. 5 that is apresentation time in accordance with the value indicated by theMPU_presentation_time information of the MMTP packets of the MPU inwhich a value indicated by mpu_sequence_number information is “4.” Inother words, there is not presented either video and the like inaccordance with the MMTP packets of the MPU in which a value indicatedby mpu_sequence_number information is “3” or video and the like inaccordance with the MMTP packets of the MPU in which a value indicatedby mpu_sequence_number information is “4.”

Further, there is not either an MPU in which a presentation time inaccordance with a value indicated by MPU_presentation_time informationis 9:00:01. 0 or an MPU in which a presentation time in accordance witha value indicated by MPU_presentation_time information is 9:00:01. 5.

In other words, when processing timings of MPUs used for presentationare determined on the basis of set MPU_presentation_time information, aproblem may occur upon eliminating a leap second. It is difficult forthe invention described in Patent Literature 1 to solve such a problem.

FIG. 10 is an illustrative diagram illustrating an example in which thereceiver 200 processes MPUs transmitted/received when a leap second iseliminated. FIG. 11 is a flowchart illustrating an example in which thereceiver 200 processes MPUs transmitted/received when a leap second iseliminated.

In the example illustrated in FIG. 10, at the same timings as in theexample illustrated in FIG. 9, MMTP packets of MPUs in which valuesindicated by mpu_sequence_number information are 1 to 10 aretransmitted. Then, when a leap second elimination flag is set for timeinformation included in the MMTP packets (Y of step S201) and a timingof eliminating a leap second falls within a predetermined time (Y ofstep S202), the receiver 200 moves to processing of step S203, and whenthe above case is not realized (N of step S201 or N of step S202), thereceiver 200 moves to processing of step S208. In the processing of stepS208, the receiver 200 executes normal processing that processes MMTPpackets used for presentation at timings based on MPU presentation timeinformation (step S208).

Specifically, in the processing of step S208, the receiver 200 decodesand presents the respective MMTP packets at timings based on calculatingresults using equation (1) and equation (2) described above, on thebasis of the MPU_presentatation_time information.

Further, the receiver 200 determines to execute processing such asdecoding and the like on the basis of timings determined in theself-running mode in processing of step S203 (step S203).

Here, the leap second elimination flag is indicated as NTPleap_indicator in FIGS. 9 and 10 and it is reset at 9:00:00. 0.Specifically, for example, a leap second elimination flag is set fortime information included in MMTP packets transmitted before 9:00:00. 0,but no leap second elimination flag is set for time information includedin MMTP packets transmitted after 9:00:00. 0.

Then, the receiver 200 determines timings of decoding MMTP packets bythe following processing in the self-running mode (step S204). In otherwords, the receiver 200 sequentially determines, at intervals inaccordance with the value of pts_offset used in the calculation ofequation (1) in processing of step S208 of an immediately prior stage,timings of decoding subsequent access units. More specifically, timingsof decoding respective MMTP packets configuring the access units aredetermined so as to decode MMPT packets configuring the access units attime intervals of a value of pts_offset, for example.

Then, the receiver 200 decodes the MMTP packets configuring therespective access units on the basis of the timings determined in theprocessing of step S204 (step S205).

The receiver 200 presents video and audio based on the MMTP packetsdecoded in the processing of step S205 (step S206). Specifically, atintervals in accordance with the value of pts_offset used in thecalculation of equation (1) in the processing of step S208 of animmediately prior stage, timings of presenting video and audio inaccordance with subsequent access units are sequentially determined.More specifically, timings of presenting video and audio in accordancewith the respective MMTP packets configuring the access units aredetermined so as to present video and audio in accordance with MMTPpackets configuring the access units at time intervals of a value ofpts_offset, for example. The receiver 200 presents the video and audioat the determined timings.

When a predetermined time has not passed from elimination of a leapsecond (N of step S207), the receiver 200 moves to the processing ofstep S204.

Then, by repeating the processing of steps S204 to S206, the MMTPpackets configuring the respective access units are decoded at timingsin accordance with the value of pts_offset in an order of sequencenumbers indicated by mpu_sequence_number information, as illustrated inthe lower side of FIG. 10. Then, video and the like based on therespective MMTP packets are sequentially presented.

It should be noted that as described above, in general, in a series ofMPUs, a value of pts_offset is not changed. Therefore, when on the basisof a calculation result in processing of step S208 of the last time anda value of pts_offset used in the calculation, MMTP packets of accessunits from the next time are used to determine decode timings, it ispossible that the respective MMTP packets are decoded at appropriatetimings and video and the like based on the MMTP packets aresequentially presented.

When the predetermined time has passed from elimination of the leapsecond (Y of step S207), the receiver 200 moves to normal processing fordetermining timings of MMTP packets used for presentation on the basisof MPU_presentation_time information (step S208).

The determination in the processing of step S207 may be performed on thebasis of whether a predetermined time has passed from a time of settingno leap second elimination flag.

Here, “the predetermined time” in the processing of steps S202 and S207is, for example, three seconds. Therefore, the processing of steps S204to S206 is repeated over three seconds each before and after a timing ofeliminating a leap second. Specifically, the processing of steps S204 toS206 is repeated, for example, from 8:59:56 that is three seconds before8:59:59 that is eliminated as a leap second to 9:00:03 that is threeseconds after 8:59:59 that has been eliminated as the leap second.

Therefore, in the present example, the receiver 200 executes processingbased on the self-running mode over three seconds before and after atiming of eliminating a leap second and executes normal processing ofstep S208 for a period other than the above. It should be noted that thetime and the number of seconds used for the description in the presentexample are illustrative, and another time and another number of secondsmay be used.

According to the present example embodiment, the receiver 200 processesMMTP packets in the self-running mode for a period in accordance with atiming of inserting a leap second and for a period in accordance with atiming of eliminating a leap second. Therefore, a problem caused due toinsertion or elimination of a leap second can be prevented fromoccurring.

Specifically, when a calculation result in calculation processing of thelast time using equation (1) and equation (2) and a value of pts_offsetused in the calculation is used to determine timings of processing fromthe next time for a period in accordance with a timing of inserting aleap second and for a period in accordance with a timing of eliminatinga leap second, it is possible that respective MMTP packets are decodedat appropriate timings and video and the like based on the MMTP packetsare sequentially presented. Respective MMTP packets are decoded atappropriate timings and video and the like based on the MMTP packets aresequentially presented, and therefore it is possible that respectiveMMTP packets are decoded in an appropriate order and video and the likebased on the MMTP packets are sequentially presented.

Further, even when the receiver 200 has received packets in accordancewith television program information and the like transmitted via acommunication line and packets in accordance with television programinformation and the like transmitted by radio waves, decoding can beperformed in an appropriate order and at appropriate timings inaccordance with a presentation order of the television programinformation and the like.

Furthermore, even when the receiver 200 has received packets inaccordance with television program information and the like transmittedvia a communication line in an order different from an order of atransmission order, decoding can be performed in an appropriate orderand at appropriate timings in accordance with a presentation order ofthe television program information and the like.

A Second Example Embodiment

Next, a decoding apparatus 20 of a second example embodiment of thepresent invention will be described with reference to the accompanyingdrawings. FIG. 12 is a block diagram illustrating a configurationexample of the decoding apparatus 20 of the second example embodiment ofthe present invention. As illustrated in FIG. 12, the decoding apparatus20 of the second example embodiment of the present invention includes adecoding unit 21 and a decode time determination unit 22.

The decoding unit 21 decodes packets transmitted from a transmitter 10(equivalent to the transmitter 100 in the first example embodimentillustrated in FIG. 1).

The decode time determination unit 22 determines timings of decoding thepackets by the decoding unit 21 on the basis of adjustment informationin accordance with adjustment of times for causing Coordinated UniversalTime to be close to Universal Time, the times being included in thepackets.

The packets include processing interval information that indicates aprocessing interval of access units that are sets including own packets,and the decode time determination unit 22 determines to sequentiallydecode the packets configuring the respective access units at intervalsin accordance with the interval indicated by the processing intervalinformation when the adjustment information indicates that the times areadjusted.

According to the present example embodiment, pieces of informationsequentially transmitted can be decoded and reproduced at appropriatetimings.

While the invention has been particularly shown and described withreference to example embodiments thereof, the invention is not limitedto these embodiments. It will be understood by those of ordinary skillin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present invention asdefined by the claims.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2015-040456, filed on Mar. 2, 2015, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

10, 100 transmitter

20 decoding apparatus

21 decoding unit

22 decode time determination unit

200 receiver

300, 500 antenna

400 communication line

600 a, 600 b multimedia information

What is claimed is:
 1. A method for receiving data, the methodcomprising: receiving a packet identification of a Moving PictureExperts Group (MPEG) Media Transport Protocol (MMTP) packet, the packetidentification identifying the MMTP packet, the MMTP packet including atleast a part of a Media Processing Unit (MPU); receiving a sequencenumber information, the sequence number information identifying the MPU;receiving a time stamp information, the time stamp informationreferencing UTC (Coordinated Universal Time); receiving a leap secondevent information, the leap second event information indicating a leapsecond event; wherein the MPU includes a video asset and/or an audioasset, a timing of presenting video and audio is based on the leapsecond event information, the video asset and the audio asset aredelivered on a plurality of transmission paths, the plurality oftransmission paths include a communication line and radio waves.
 2. Themethod of claim 1, wherein the video asset is delivered to a firstbuffer based on the packet identification of the video asset, the audioasset is delivered to a second buffer based on the packet identificationof the audio asset, and a control message is delivered to a thirdbuffer.
 3. The method of claim 2, wherein the control message comprisesinformation used to control the receiving.
 4. A receiving device forreceiving data, the receiving device comprising: a receiver configuredto: receive a packet identification of a Moving Picture Experts Group(MPEG) Media Transport Protocol (MMTP) packet, the packet identificationidentifying the MMTP packet, the MMTP packet including at least a partof a Media Processing Unit (MPU); receive a sequence number information,the sequence number information identifying the MPU; receive a timestamp information, the time stamp information referencing UTC(Coordinated Universal Time); receive a leap second event information,the leap second event information indicating a leap second event;wherein the MPU includes a video asset and/or an audio asset, a timingof presenting video and audio is based on the leap second eventinformation, the video asset and the audio asset are delivered on aplurality of transmission paths, the plurality of transmission pathsinclude a communication line and radio waves.
 5. The receiving device ofclaim 4, wherein the video asset is delivered to a first buffer based onthe packet identification of the video asset, the audio asset isdelivered to a second buffer based on the packet identification of theaudio asset, and a control message is delivered to a third buffer. 6.The receiving device of claim 5, wherein the control message comprisesinformation used to control the receiving.
 7. A non-transitory computerreadable medium storing a program which, when executed, causes acomputer to execute: receiving a packet identification of a MovingPicture Experts Group (MPEG) Media Transport Protocol (MMTP) packet, thepacket identification identifying the MMTP packet, the MMTP packetincluding at least a part of a Media Processing Unit (MPU); receiving asequence number information, the sequence number information identifyingthe MPU; receiving a time stamp information, the time stamp informationreferencing UTC (Coordinated Universal Time); receiving a leap secondevent information, the leap second event information indicating a leapsecond event; wherein the MPU includes a video asset and/or an audioasset, a timing of presenting video and audio is based on the leapsecond event information, the video asset and the audio asset aredelivered on a plurality of transmission paths, the plurality oftransmission paths include a communication line and radio waves.
 8. Thenon-transitory computer readable medium of claim 7, wherein the videoasset is delivered to a first buffer based on the packet identificationof the video asset, the audio asset is delivered to a second bufferbased on the packet identification of the audio asset, and a controlmessage is delivered to a third buffer.
 9. The non-transitory computerreadable medium of claim 8, wherein the control message comprisesinformation used to control the receiving.